On Oblique Random Forests
نویسندگان
چکیده
Abstract. In his original paper on random forests, Breiman proposed two different decision tree ensembles: one generated from “orthogonal” trees with thresholds on individual features in every split, and one from “oblique” trees separating the feature space by randomly oriented hyperplanes. In spite of a rising interest in the random forest framework, however, ensembles built from orthogonal trees (RF) have gained most, if not all, attention so far. In the present work we propose to employ “oblique” random forests (oRF) built from multivariate trees which explicitly learn optimal split directions at internal nodes using linear discriminative models, rather than using random coefficients as the original oRF. This oRF outperforms RF, as well as other classifiers, on nearly all data sets but those with discrete factorial features. Learned node models perform distinctively better than random splits. An oRF feature importance score shows to be preferable over standard RF feature importance scores such as Gini or permutation importance. The topology of the oRF decision space appears to be smoother and better adapted to the data, resulting in improved generalization performance. Overall, the oRF propose here may be preferred over standard RF on most learning tasks involving numerical and spectral data.
منابع مشابه
Oblique Random Forests for 3-D Vessel Detection Using Steerable Filters and Orthogonal Subspace Filtering - Supplementary Material
متن کامل
Classifying Very-High-Dimensional Data with Random Forests of Oblique Decision Trees
The random forests method is one of the most successful ensemble methods. However, random forests do not have high performance when dealing with very-high-dimensional data in presence of dependencies. In this case one can expect that there exist many combinations between the variables and unfortunately the usual random forests method does not effectively exploit this situation. We here investig...
متن کاملOblique Random Forests for 3-D Vessel Detection Using Steerable Filters and Orthogonal Subspace Filtering
We propose a machine learning-based framework using oblique random forests for 3-D vessel segmentation. Two different kinds of features are compared. One is based on orthogonal subspace filtering where we learn 3-D eigenspace filters from local image patches that return task optimal feature responses. The other uses a specific set of steerable filters that show, qualitatively, similarities to t...
متن کاملJoint 3-D vessel segmentation and centerline extraction using oblique Hough forests with steerable filters
CONTRIBUTIONS We propose a novel framework for joint 3-D vessel segmentation and centerline extraction. The approach is based on multivariate Hough voting and oblique random forests (RFs) that we learn from noisy annotations. It relies on steerable filters for the efficient computation of local image features at different scales and orientations. EXPERIMENTS We validate both the segmentation ...
متن کاملSupplementary file for Robust Visual Tracking Using Oblique Random Forests
In this document we first provide the derivation for incremental learning of PSVM parameters, which is used to update every decision node in our online Obli-RaF. Next, we provide a detailed analysis of various parameters of the proposed tracker, and demonstrate the merits of our Obli-RaF. Finally, we provide more detailed results of our approach on the OTB51, OTB-100 datasets and VOT2016 under ...
متن کامل